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The selection and development of six tests measuring 
the ability to receive and interpret nonverbal communications are 
described, as is an attempt to gather evidence of their value as 
predictors of success in two occupations reguiring high levels of 
.interpersonal srills — psychology and counseling^ The tests were: (1) 
V Inter-Person Perception Test: (2) Test of Implied Meanings; (3) 
wpONS-audio from the Profile of Nonverbal Sensitivity; (^) 
"PONS*visual'« from the Profile of Nonverbal Sensitivity: (5) Photo 
Sequence Comprehension Test: and (6) Photo Classification Test (PCT) • 
The 106 subjects were graduate students and had taken also the Group 
Embedded Figures -Test (GEFT) and the Graduate Record Examination 
(GPE) or the Miller analogies Test. The principal criterion measure 
was a faculty rating form used to rate the student. on a nine-point 
scale in: aca.demic work, interpersonal relations with pee;:s, faculty, 
and clients: personal qualities: and predicted effectiveness as a 
psychologist or counselor. Ratings wiere collected several months 
-after testing. The PCT seems to have been the most effective 
predictor, but it was uncorrelated with the GRE end only weakly 
correlated with the GEFT, as were most of the other nonverbal 
communication tests. (Ruthor/RL) 
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ABSTRACT 



Six tests of nonverbal communication skills were Investigated In an attempt 
to Improve prediction of success for psychologists and counselors. The subjectfl 
were graduate students at two different schools; the criterion variables were 
faculty members' judgments of the students' academic work, Interpersonal rela- 
tions, personal characteristics, and "predicted effectiveness" In the profession. 
Faculty ratings were collected several months after students were tested. One 
of the six nonverbal communication tests predicted faculty ratings of several 
characteristics at both schools. This test was uncorrelated with the Graduate 
Record Examinations and only weakly correlated with the Group Embedded Figures 
Test , as were most of the other nonverbal communication tests. 
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NONVERBAL COMMUNICATION TESTS 
AS PREDICTORS OF SUCCESS IN 
PSYCHOLOGY AND COUNSELING 

This project grew out of an attempt to Improve the prediction of pro- 
fessional success by measuring skills that are not measured by traditional 
aptitude tests. Most traditional aptitude tests focus on verbal and mathe- 
matical skills, and, to a lesser extent, spatial and abstract reasoning skills. 
While these skills are Important In many professions, there Is another group 
of skills that are also of great Importance In many professions: Interpersonal 
skills. 

The difficulties of measuring Interpersonal skills In any reasonably 
objective and efficient way are enormous. Interpersonal behavior Is an Inter- 
active process consisting of complex responses to complex stimuli. The stimuli 
.ire difficult to standardize and the responses are difficult to evaluate. One 
promising approach to the problem is that of role-playing simulations (Levine 
and McGuire, 19b8, 1970), but these measures are expensive to administer and 
score. Even if they were to become much more widely used rh^n they have been, 
there would still be a need for measures that can be administered to several 
examinees at once and scored by machines or by persons without special training. 

Most, if not all, interpersonal activities involve some form of communi- 
cation* It seems reasonable to assume that a person cannot be effective at an 
occupation requiring a high level of interpersonal skill unless that person can 
communciate effectively. Communication in our culture usually involves the use 
of words, but it almost always includes many nonverbal behaviors as well. Tests 
of verbal skills are widely used in occupational selection and counseling. 
Tests of nonverbal communications skills are not. 
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Communle«tlDnt gkllli can be divided into two cypea: sending and receiving. 
In the case of nonverbal communication, the receiving skills are much easier to 
measure than the sending skills; the encoded messages can be standardized in the 
form of pictures and recordings, and the examinee's responses can be obtained 
In a multiple-choice format. For this reason, most existing tests of nonverbal 
communications skills are tests of receiving skills, rather than sending 
skills. 

Some research in this area has been done by Rosenthal and his colleagues 
(Rosenthal, et al > 1974, 1979) using their Profile of Nonverbal Sensitivity 
(PONS), which yields separate subscores for various combinations of face, body, 
and voice communications. They found that skill at interpreting nonverbal 
communication, as measured by the PONS, tended to increase with age up 
to the late teens. "Impaired groups"— psychiatric patients and alcoholic 
patients— did not rerform as well as normal adults. More relevant to thv? 
present study were the differences between occupational groups. Actors 
tended to do quite well, but clinical psychologists performed only about as 
well on the PONS as college students, and teachers and business, executives 
performed no better, on the average, than high school students. One attempt 
to use a portion of the PONS to predict occupational success was reported by 
McClelland and Fiske (197A). They used the twenty random-spliced audio items 
as a separate test, with an adjustment based on the test-taker's age.* The 



*The adjustment added to the test-taker's score one percent of the number of 
questions missed for every year of age over 35. Thus, a person of 45 who 
answered 12 of the 20 questions correctly would receive an adjusted score of 
12 plus 10 percent of 8, or 12.8. 
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subjects were human service workers In a large Eastern city. The criterion 
variable was a supervisor ""s rating of the worker as "outstanding** or '*average**; 
26 of the 51 subjects were rated as **out8tanding .** The age-adjusted scores on 
the random-spliced PONS items correlated .A5 with the supervisors' ratings, 
Othir correlational studies cited by Rosenthal, et al . (1979, p, 372) reported 
positive correlations between total scores on the full PONS (220 items) and 
supervisors' ratings of the professional skills of teachers and of clinicians • 

Is it reasonable to expect tests of nonverbal receiving skills to be 
useful as indicators of interpersonal skills in general? The answer to this 
question is yes, If either of the following two conditions is true; 

!• Much of the variation between persons in interpersonal effectiveness 
is directly caused by differences in ability o receive and interpret 
nonverbal messages, ot_ 

2. Nonverbal receiving skills are highly correlated with other inter- 
personal skillst 

If either of these conditions is true, tests of the ability to receive and 
interpret nonverbal communications could be valuable instruments in the selec- 
tion, guidance, and placement of persons into or out of occupations requiring 
high levels of interpersonal skills—particularly if the tests are objectively 
scored and suitable for group administration. The project described in this 
report involved the selection and development of several such measures and an 
attempt to gather evidence of their value as predictors of success in two such 
occupations • 
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METHOD 

Tegt ■•lactlon and devlopinftnt 

Thtrt are at least four Iroportaht ways in which people transmit nonverbal 
measagea in situations associated with their work: tone of voice, facial 
expreasion, body position, and the use of interperbonal apace. The tests 
investigated as part of this project Involved all of these four channels of 
coimnunlcatlon, cither separately or In combination with each other. 

The first phase of this project consisted of the selection of tests 
that seemed promising as predictors of success In a wide range of occupations 
involving interperaonal skills, including the modification of one existing test 
and the development of two new tests.. In reviewing tests purporting to measure 
interpersonal skills, the investigator excluded tests that appeared to depend 
heavily on verbal skills, tests based on stimulus materials that appeared 
to be highly contrived or artificial (e.g., cartoons or obviously posed 
photographs), and tpsts that required the examinee to have knowledge, training, 
or experience in a particular occupation. The tests selected for inclusion in 
the study were the Inter-Person Perception Test (Heussenstamm and Hoepfner, 
1969), the Test of Implied Meanings (Sundberg, n.d.) and two tests from the 
Profile of Nonverbal Sensitivity (Rosenthal, et al, 197A, 1979). 

The Inter*Person Perception Test (IPPT) is a purely nonverbal test that 
focuses entirely on the face. Each item consists of a photo of a person's 
face, followed by four photos of a second person's face. The examinee is 
instructed to "mark the letter for one of the four faces on the right that Is 
thinking or feeling the same as the f:*rst person." Both persons in an item are 
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of tha ■•DC r«e« and itx. The teat eontalna forty itana, five for each race/aax 
combination of both aexea and four raeaa: White (Euro-American), Black (Afro- 
American), Japaneae (Japaneae-American) , and Chicano (Mexican-American). This 
teat waa used without modification. 

"^h" Teat of implied Maaninaa ii a teat intended to meaaure a peraon'a 
ability to perceive and interpret the messagea conveyed by inflection and tone 
of voice in apoken language. The test is administered by tape recording. Each 
Item consists of a short sentence or portion of a sentence, spoken in such a way 
as to Imply more than the words themselves indicate. The examinee must choose 
the one of four written phrases that expresses the additional meaning implied by 
the way the sentence is spoken. The original version of this test consisted of 
forty items, all read by narrators. The version used in this study consisted 
of twenty items from the original test, plus ten items created from tape 
recordings of "natural" speech in meetings and discussion groups. In con- 
structing these ten newly created items, the investigator determined the 
correct Implied meanings from portions of the tape-recorded discussion that 
were not included in the brief excerpt played for the examinee. 

"^^^ Profile of Nonverbal Sensitivity (PONS) consists of several different 
types of Items. Each item is formed from a videotape recording of a young 
woman, simply and neatly dressed, role-playing a common situation. The test 
taker must choose the one of two brief phrases that correctly describes what 
the woman la doing (e.g., "expressing gratitude"). The original PONS consists 
of 220 Items presented In movie form. The version used In this study consists 
of two 40-ltem tests. One test ("PONS-audlo") Is administered by tape recording. 
Each audio Item Is based on a two-second fragment of the audio portion of the 
recording, with' the words made unintelligible. In half the Items the tape 
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rtcordtng hat bf«n ''content Cllttrtd** to blur tho tpocfch by rtmovlng the high 
frtquonclttt In the othtr half of tha Itami th« tape haa boan aubjactad to 
"random apllclng**, l«a«, tha tapa la cut Into vary abort fragmantai which ara 
randomly ahufflad and raaaaamblad, producing a garbled varalon of the original 
apaach excerpt. The aacond teat (*'PONS-vlaual**) la baaed on atlll photographa 
made from the PONS videotape. Half the photoa ahov only the voman^a face; the 
other half ahov her body from the ahouldera to the kneea* She la dreaaod 
Identically In all the photoa. The proceaa of development of the PONS la 
deacrlbed In detail by Roaenthal, at al (1979). 

Two additional teats were developed for Ula project by the Investigator 
with the aid of a consultant, an expert In the Interpretalon of nonverbal 
communication.* Both those tests are based on photographs, but, unlike those 
In the IPPT and the PONS, the photos In these tests all show an Interaction 
Involving two or more persons. 

The Photo Sequence Comprehension Test Is similar to a reading comprehension 
test In format. However » Instead of reading a paragraph, the examinee looks 
at a sequence of 10 or 12 photographs of two or more people In a situation 
Involving some degree of Interpersonal Interaction (e.g., a business discussion 
In an office). Each sequence of photos Is followed by four multiple-choice 
questions about the actions, thoughts, and feelings of the people pictured. 

To develop the Photo Sequence Comprehension Test the Investigator first 
obtained the permission of the persons In the photos to photograph them whllj 
they engaged In their normal workday activities. (Exception: one of the five 
sequences was role-played, although the persons Involved were all acting in 

*Dr. Sylvan Tomklns 



r«X«t th9f lecyffeoMvd to.) Th« inv«atig«tor cook « ierles of Approxiouicely 
60 phoeoi» uilni available light to avoid the Incruilon of a fiaah. After the 
phocoa were printed » the Inveatigator arranged them In aequenee for the conaul- 
tant» who "read" the nonverbal meaaagea they contained. The Inveatlgator then 
auimarUed the verbal content of the interaction for the conaultant, and the 
conaultant aelected a aequenee of 10 or 12 photoa in which the nonverbal 
meaaagea told the atory of what had happened. The inveatlgator and conaultant 
then decided what queatlona to aak about the photo aequenee. Thla proceaa waa 
repeated for each of the five aequencea on the teat, with four queatlona baaed 
on each aequenee. 

The Photo ClaeHl'jcation Teay U an entirely nonverbal teat. Each Item- 
aet eonslsta of nine phofua ahowing the aame two people in the aame a*ttlng 
(••R., a man handing jome papera to a woman alttlng behind a typewriter). 
On the left pase aru alx phctoa, claaalfled Into two criterion grcupa labeled 
A and B. On the right page are throe more photoa. The examlnee'a taak la to 
classify each of these three additional photoa aa belonging with group A or 
group R. The examinee Is not told the basis for the elaaalf leation. but must 
Infer It b\ looking at the photos In groups A and B. In fact, the teat la 
constructed so that the feelings expressed by the people in the pictures form 
the basis for the classification. Other aspects of the situation (e.g. the 
placement of physical objects) vary in ways that do not consistently distinguish 
between group A and group B. 

Photo Classification T^t was developed by having the same peraons 
role-play a common situation several times. Two different verslona of each 
situation were role-played. The persons and the setting were the same in 



11 



-i- 

both vsrttoni, but the "Mrlpi", t.«., the InitrueHeni given ©rilly to the 
ro]e«plAyert, wag changed, lo m t© imply a different emotional content in the 
two veraiona. (Exceptioni one of the five item-aeta was nade up of photea taken 
durtng an actual buatneaa meeting and aeleeted with the aid of the consultant 
who aeaiated in the development of the Photo S»HUBnce CoworahenBion Teat .) 

Suh1»cf 

The eubjecta for the atudy were atudenta in graduate proRrans in paychology 
and counaeling at two different tnatitutiona. School I ia a graduate achool 
located at a major atate unlveralty. The achool haa Ph.D and Pay.O. programs 
in clirical paychology and in achool paychology. Hoat of the atudenta are 
full-time graduate atudenta. School 2 ia a amall college which haa a maater'a- 
degree program in academic counseling. Claaaea in this program meet in the 
evening, and many of the atudenta are not full-time atudenta. Participation 
at both achoola waa voluntary, and the atudenta were not paid. Participation 
conalated almply of taking the teata and algnlng a form allowing the investigator 
CO have acceaa to the student's academic rccorda and to dlacuaa the atudent'a 
profeaaional work with faculty membera. At School 1 the atudenta were released 
early from a regular class to take the teata. At School 2 the teata were given 
during a regular clasa period. 

loatrumente 

In addition to the nonverbal communication teata, the atudents partici- 
pating In the study took the Crouo Embedded Fieuras Test (Oltman et al . 1971). 
This test Is nade up of problems that require the test-taker to locate a simple 
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geometric figure that Is embedded In a complex geometric pattern. The test 
consists of a practice set of seven problems, followed by two separately timed 
sets of nine problems each* 

Additional predictors were available to the Investigator for many of the 
students at School 1, where many of the students had taken either the Graduate 
Record Examlngt-lon or the Miller Analogies Test . The students' scores on these 
tests were Included In the data analysis. 

The principal criterion measure for the study was a faculty rating form. 
This form was a modified, expanded version of a portion of an existing form 
used for practlcum supervisors' evaluations at School 1. The form called for 
the faculty member to rate the student on a nine-point scale In eacn of several 

_ca^t^e^aiJ.esjL.academ±c^work4-~J.at«^ 

clients; several personal qualities (clarity, discretion, enthusiasm. Initiative, 
organization, poise, responsibility, sensitivity, tact, warmth); and, finally, 
"predicted effectiveness" as a psychologist or counselor. On the rating form 
each variable was Identified only by a single word or brief phrase; no additional 
descriptive Information was provided. In addition, the rating scale for each 
specific variable contained an extra space labeled "cannot rate student". The 
schools were asked to distribute rating forms for each partlclptlng student to 
all faculty members who had taught the student. • Faculty members were asked to 
rate only those students they felt they knew well enough to rate meaningfully. 
Faculty ratings were obtained for 66 students at School 1 and AO students at 
School 2. The number of faculty members rating each student at School 1 varied 
from one to twelve. At School 2 each student was rated by two faculty members. 

Procedure 

The tests were administered on several occasions. At School 1 there were 
testing sessions In Febrcsry and March of 1977, September of 1977, and September 
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of 1978. At School 2 there were testing sessions In October of 1977 and 
September of 1978. Faculty ratings were collected at both schools In the 
spring of 1978 and 1979. The time Interval between a student's testing session 
and the collection of faculty ratings for that student varied from nine months 
to 21 months. None of the faculty members at either school had access to any 
Information about the students' performance on the tests. 

The tests were administered to groups ranging In size from seven to 
approximately thirty students. The investigator began by asking the students 
to read a description of the research study and to sign a statement of Informed 
consent. The description of the research study described the tests as being 
"based on pictures and sound recordings" and "intended to measure skills that 
are*WC^Wasure The^flr^srTe 
administered was the Group Embedded Figures Test * It was followed by the 
Photo Classification Test, the Test of Implied Meanings , the Photo SeouRnce 
Comprehen sion Test , the PONS-audio, the PONS-visual, and the Inter-Person 
Perception Test , in that order. 

Because the students were not all rated by the same faculty members, 
the effects of the raters' differing standards had to be removed frow the 
criterion data. This step was accomplished by a regression procedure which 
assumes that each Individual rating of a student by a teacher can be expressed 
as the sum of a student-quality parameter, a teacher-leniency parameter, and a 
random variable. In mathematical notation, let x^^ represent the rating of 
student 1 by teacher j . Then 

."ij "^i^^'^j r^ij 

where a^ is the student-quality parameter for student 1 , bj is the teacher- 
leniency parameter for teacher j , and e^^ is the remainder, assumed to be the 
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result of random variation. The a^ parameter Is the average of the ratings that 
student 1 would have received If all students had been rated by all teachers. 
The regression procedure estimates this parameter for each student. The pro- 
cedure was done separately for the two schools and separately for each variable 
on the rating form. The estimated student-quality parameters (the a^ In the 
model) were then used as criterion measures. Wherever the term "faculty ratings" 
appears in the Results section of this report. It refers to these estimated 
student-quality parameters. 
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RESULTS 

The key question to be answered by the data is whether or not any of the 
tests effectively predicted the judgments that the faculty members made several 
months later, after working closely with the students. This question can best 
be answered by an examination of the correlations between the test scores and 
the faculty ratings. The sample size for these correlations varies somewhat, 
because some faculty members did not rate all of the students on all of the 
Variables and because a few students did not take all of the tests. The sample 
sizes for the correlations range from 57 to 66 at School 1 and from 35 to 40 
at School 2. For "predicted effectiveness," which was Intended as the main 

criterion measure, the sample sizes range from 64 to_. 66„at_SghQQl l. and^f rom, . 

38 to 40 at School 2. (The minimum correlations necessary for statistical 
sl?gnlf Icance at the five percent level are therefore approximately .24 at 
School 1 and .33 at School 2.) 

Table 1 shows the correlations of the tests with the faculty ratings, kn 
Inspection of these correlations reveals some Interesting results. First, the 
correlations for School 2 are generally larger than those for School 1. This 
fact may be a consequence of differences In the reliability of the ratings 
(discussed below). In particular, the ratings of academic work at School 2 
seem to have been predictea quite well by three of the nonverbal communication 
tests. 

The Photo Classification Test seems to have been the most effective pre- 
dictor. In fact, for many of the ratings, including "predicted effectiveness," 
it was the only effective predictor. In Table 1 there are only five combina- 
tions of predictor and Criterion for which the correlation is at least .20 at 
both schools. In four of these five combinations, the predictor is the Photo 
Classification Test . 
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Table 1. Correlations of tests with faculty ratings. 
(Upper numbers refer to School 1, lower 
numbers refer to School 2.) 



Academic Interpersonal Relations with ... 

Work super- patients/ faculty peers Clarity Discretion Enthusiasm 
visors students 



Inter-Person 


.06 


* 


. uo 


- IS 


1/. 




.09 


-.08 


Perception Test 


.41 


-.14 


-.15 


--21 


-.18 


*36 


-.16 


-.01 


Test of Implied 


-.10 


* 


.18 






• UO 


• Uj 


-.11 


Meanings 


.04 


.06 


-.07 


.21 




• Hi 


m 

• Ul 




Profile of Nonverbal 


.01 


* 


-.01 


.03 


.05 


-.09 


.06 


.07 


Sensitivity-audio 


.17 


-.26 


-.20 


-.22 


-.16 


.05 


-.01 


-.09 


Profile of Nonverbal 


.02 


It 


.06 


-.10 


-.09 


-.07 


-.13 


-.02 
.04 


Sensitivity-visual 


.38 


.25 


-.04 


.44 


.24 


.25 


.22 


Photo Sequence 


"""Toi 




-.10 


.03 


-.02 


.26 


.oT" 


-.08 


Comprehension Test 


.07 


-.34 


-.16 


-.41 


-.32 


.08 


-.40 


-.18 


Photo Classification 


.21 


* 


.19 


.16 


.10 


.24 


.07 


.33 


Test 


.45 


.02 


-.16 


.08 


-.17 


.37 


.15 


.01 


Group Embedded 


.17 




-.12 


.01 


.03 


.05 


-.04 


-.12 


Figures Test 


.33 


-.05 


-.23 


.02^ 


-.20 


.24 


.14 


-.28 



*This variable was rated at school 2 only. 
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Table 1 (cont.) 



Inter-Person 
Perception Test 

Test of Inplled 
Meanings 

Profile of Nonverbal 
Sensltlvlty-audlo 

Profile of Nonverbal 
Sensltlvity-vlsual 

Photo Sequence 
Comprehension Test 

"PhlltrClisWlcafion" 
Test. 

Group Embedded 
Figures Test 



i-niLia* 


Organi- 




Responsi- 


Sensi- 






Predi( 


tlve 


zation 


Poise 


bility 


tivity 


Tact 






••11 


-.01 


-.03 


-.05 


.00 


.01 


-.13 


•03 


.04 


•16 


.04 


.19 


-.03 


-.23 


-.23 


-•U4 




-.OJ 


.12 


-.04 


.03 


.05 


.07 


.07 


.00 


.27 


.21 


.20 


.27 


.22 


.07 


• 10 


.10 


•11 


-.07 


.11 


-.06 


.02 


-.07 


.00 


-.12 


-•08 


-.03' 


-.21 


-.03 


-.09 


-.10 




fli 

-.Ul 


•01 


-.13 


-.10 


-.10 


-.03 


-.03 


-.08 


•.08 


• J/ 


91 




.10 


.32 


.16 


.14 


-.11 


-.06 


-.02 


-.10 


.00 


-.09 


-.03 


.07 


-.06 


-.13 


-.08 


-.10 


-.23 


-.26 


-.31 


-.20 


~T2'r™"' 

.25 




■""'.05 


' .iT 


'""Ti2 


" .03 


.01 


.24 


.30 


.28 


.36 


.23 


.11 


-.04 


.26 


.06 


.10 


-.07 


-.02 


-.10 


-.07 


-.15 


.04 


-.22 


.17 


-.12 


.16 


-.25 


-.01 


-.30 


.07 
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The PONS-vlsual and, to a lesser extent, the Test of Implied Meanings seem 
to have predicted several of the ratings effectively at School 2, but not at 
School 1. The PONS-audlo and the Photo Sequence Comprehension Test failed to 
predict any of the ratings (except possibly In the negative direction, I.e., 
with higher scorers on these tests receiving poorer ratings). 

The Group Embedded Figures Test predicted some of the faculty ratings, 
particularly at School 2. The test correlates positively with ratings of 
academic work, "clarity," and "organization"; negatively with ratings of 
Interpersonal relations with patients or students. Interpersonal relations 
with peers, "sensitivity," and "warmth." 

Table 2 shows the range of possible scores, the mean, standard deviation, 
and reliability coefficient* for each test, and the Intercorrelatlons among the 
tests. The data In Table 2 are based on larger samples than those In Table 1, 
because they Include some additional students who took the tests but were not 
rated by any faculty members. The sample sizes range from 80 to 83 at School 1 
and from A8 to 50 at School 2. Obviously, the tests do not correlate highly 
with each other; the highest correlation between any two tests Is .39, and only 
five of the 42 correlations between tests are larger than .20 In size. One 
surprising finding Is the very low reliability of the PONS tests, especially In 
light of the success of the visual PONS In predicting several of the faculty 
ratings at School 2. 



*For the two PONS tests these are KR-20 (or "alpha") coefficients. For the 
other tests they are split-halves or. In the case of the Photo Classification 
Test^ split-thirds coefficients » corrected with the Spearman-Brown formula. 
Attempts to use the split-halves method vlth the PONS tests yielded correla- 
tions less than zero In two of the four cases » even though the splits were 
made for maximum similarity between the two half-tests. 
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Table 2. 



Test score data. (Upper numbers refer to School 1; 
lower numbers refer to School 2.) 





IPPT 


Implied 
Meanings 


PONS- 
audio 


PONS- 
visual 


Photo 
Sequence 


Photo 
Classification 


uUlUCuUCU 

Figures 


Possible score range 


0-40 


0-30 


0-40 


0-40 


0-20 


0-15 


0-18 


Chance score 


10 


7.5 


20 


20 


5 


7.5 


* 


Maan 

nean 


25.1 
23.4 


19.8 
18.9 


25.8 
'26.0 


29.8 
29.8 


12.4 
11.6 


9.2 
8.0 


12.4 
9.6 


Standard deviation 


3.3 
3.6 


3.9 
3.8 


3.4 
2.6 


2.7 
2.7 


2.5 
2.2 


3.1 
3.0 


4.6 
5.3 


Reliability coefficient 
Intercorrelatlon with 


.35 
.40 


.68 
.75 


.46 
.07 


.19 
.28 


.39 
.37 


.85 
.74 


.83 

.87 


Inter-Person 
Perception Test 




-.02 
.16 


-.08 
.28 


-.05 
.06 


.04 
.39 


-.13 
.12 


.02 
.02 


Test of Implied 
Meanings 






-.04 

„ .1,3 


.08 
.25 


.09 

.09 


-.12 

- .29 


.32 

.22 



.03 .11 .16 



PONS-vlsual 



Photo Sequence 
Comprehension 

Photo Classification 



*The Group Embedded Figures Test: is not a multiple-choice test. 
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PONS-audlo .n? n«; 

'05 ,11 -.05 



.11 



■•02 -.21 -.13 

-•18 -.09 -.10 

.02 .33 
•18 .11 

.10 
.',0 



I 

M 
I 



0 

■J 
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The scores on most of the tests were slightly higher at School 1 than at 
School 2. The data do not suggest any "floor" or "celling" effects. Only the 
Photo Classification TPftt and the Group Embedded Figures Teat „ere "speeded" In 
the conventional sense, I.e., so that a slow-working examinee would not reach 
the last Items on the test. This speededness probably accounts for the fact 
that several students scored below chance on the Photo Classification Teat .* 
Two Interesting features of the scores on the Group Embedded Figures Tpst are 
not reflected In the data In Table 2. The score distribution was very flat or 
possibly blmodal, and there was a noticeable practice effect between the two 
separately timed halves of the test. 

Table 3 shows the Inter-rater reliability of the faculty ratings. The 
numbers In Table 3 represent the proportion of the variation In the ratings that 
Is accounted for by differences between students. Almost without exception, 
the reliability of the ratings is higher at School 2 than at School 1. This 
difference In reliability of the ratings may account for the fact that the 
cotielatlons between test scores and faculty ratings tended to be larger at 
School 2 than at School 1. The most reliable ratings were the ratings of 
academic work at School 2, and these were the ratings best predicted by the 
nonverbal communication tests. Most of the reliabilities at School 1 were In 
the range of .40 to .50. Using these reliability estimates to "correct for 
unreliability In the ratings" (I.e., to estimate the correlation between the 
test and a perfectly reliable rating) would Increase the reported correlations 
for School 1 by aproxlmately 40 to 60 percent. For example, the correlation 
between the Photo Classification Test and the "predicted effectiveness" rating ' 
would Increase from .24 to .37. 



*It may also account for the surprisingly high reliability of the Photo 
Classification Test, which Is probably partly spurious, since the split- 
thirds were not separately timed. 

2d 



-18- 

Table 3. Inter-rater reliability 
of faculty ratings. 



School 1 School 2 

Academic work .52 .78 

Interpersonal relations with 

supervisors * ,54 

patients /students, .25 .47 

faculty .36 .62 

peers .56 .55 

Clarity .45 .57 

Discretion .43 .65 

Enthusiasm .40 .61 

Initiative .40 .58 

Organization " .51 .59 

Poise .37 .48 

Responsibiity .47 .55 

Sensitivity .40 .53 

Tact .41 .56 

Warmth .27 .47 

Predicted effectiveness ,42 .60 



*Rated only at School 2. 



Table 4 shovs the Intercorrelatlons of the faculty ratings. The correla- 
tions tend to be positive, with only a fev exceptions. The highest correlations 
are where they might be expected, e.g., between "academic work" and "clarity," 
between "enthusiasm" and "initiative," and between "organization" and "responsi- 
bility." Similarly, the lowest correlations are where they might be expected, 
e.g., between "academic work" and "warmth." The faculty at School 1 seem to 
have placed a greater emphasis on intellectual qualities in rating "predicted 
effectiveness" than did the faculty at School 2, as indicated by the correlations 
in the last row of Table A. However, "predicted effectiveness" correlates at 
least .45 with all the other ratings at both schools and thus seems to fulfill 
its Intended purpose as a single overall criterion measure. A word of caution 
may be in order, though: the rating most highly correlated with "predicted 
effectiveness" is "interpersonal relations with faculty." Nevertheless, it 
seems reasonable to use "predicted effectiveness" as an overall criterion 
measure wherever a single measure is necessary to avoid burdening the reader 
with huge quantities of statistical information. 

Several of the tests are constructed in such a way that meaningful sub- 
scores can be defined. The Photo Sequence Comprehension Test and the Photo 
Classifi cation Test each consist of five separate sets of items, with each item 
set based on a single collection of pictures. In the Inter-Person Perception 
Test, the items can be classified according to the race or sex of the persons 
pictured. The Test of Implied Meanings can be subdivided into three groups of 
items on the basis of the speakers in the recording; only the items in the 
third group are based on excerpts of "natural" conversation. The audio-PONS 
can be subdivided into "content-filtered" and "random-spliced" speech excerpts. 
The visual-PONS can be subdivided into pictures showing the face only and those 



Tabl« 4i Intercorrolntlons of faculty rating8< 
(Uppor numbers aro for School 1; 
lower numbers, for School 2<) 

iSy ! Academic 

Work 



Interpersonal * Interpersonal 

-supervisors tSO •supervisors 



Interpersonal 


.OS 


* 


InterpecBonal 




















•patients /students 


.07 


.55 


-petlents /students 


















Interpersonal 


.32 


* 


.71 


Interpersonal 


















•faculty 


.37 


.79 


.45 


-faculty 




















Interpersonal 


.31 


* 


.58 


.70 


Interpersonal 
















"peers 


-.02 


.67 


.68 


.67 


-peers 


















Clarity 


.61 


« 


,49 


.57 


.38 


Clarity 


















.83 


.34 


.12 


10 


-.01 


















Discretion 


.37 


* 


.53 


M 


.46 


.57 


Discretion 














.11 


.69 


.25 


.65 


.56 


.08 
















Enthuslastt 


.58 




.62 


.65 


.37 


.54 


.37 


Enthuslam 












.23 


.53 


.54 


.40 


.34 . 


.26 


.10 














Initiative 


.62 


* 


.59 


.61 


.23 


.39 


.47 


.77 


Intlatlve 








« 


.37 


.37 


.35 


.29 


.12 


.33 


.00 


.84 












Orgsnleatlon 


.76 


* 


.37 


.45 


.14 


.57 


.64 


.57 


.84 


Organliatlon 








.77 


.50 


.08 


.51 


.07 


.65 


.22 


.41 


.53 










Poise 


.31 


* 


.54 


.55 


.45 


.57 


.67 


.37 


.40 


.46 


Folse 








.41 


.49 


.37 


.51 


.45 


.42 


.32 


.31 


.35 


.36 










.60 




.55 


.59 


.34 


.54 


.68 


.64 


.77 


.84 


.59 


ReaponsiblUty 




.77 


.60 


.23 


.54 


.16 


.70 


.31 


.48 


.62 


.88 


.51 






SeniUlvlty 


.33 


* 


.64 


.51 


.60 


.52 


.57 


.39 


.32 


.39 


.72 


.59 


Sensitivity 




.12 




.56 


.52 


.48 


.21 


.43 


.37 


.27 


.10 


.34 


.22 


Tict 


.34 


t 


.59 


.62 


.50 


.45 


.84 


.33 


.43 


.55 


.71 


.59 


.69 Tact 




.15 


.67 


.51 


.84 


.72 


.21 


.71 


.28 


.12 


.17 


.48 


.23 


.71 


yarnth 


.09 


* 


.49 


.55 


.68 


.21 


.43 


.41 


.26 


.12 


.47 


.27 


.58 .58 




-.07 


.68 


.74 


.56 


.77 


-.07 


.46 


.46 


.20 


.04 


.34 


.17 


.64 .64 


Predicted 


.61 


« 


.69 


.76 " 


.48 ' 


.71 


.69 


.73 


.67 


.65 


.66 


.70 


.55 .66 


effectiveness 


.47 


.76 


.70 


.74 


.62 


.47 


.51 


.60 


.51 


.46 


.54 


.61 


.61 .71 



ARsted only at School 2. 
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showing the body only. Both PONS tests can be divided Into subscores on the 
basis of the almllarlty of the correct and Incorrect options, In terms of the 
feeling expressed— positive or negative— and the status of the speaker In 
reaction to the person spoken to — dominant or subordinate. (See Rosenthal, 
et_al., 1979, pp. 29-30.) 

Table 5 presents the correlations of these subscores with the faculty 
ratings and also the mean percent-correct for each subtest, at each of the two 
schools. For the Inter-Person Perception Teat and the Test of Implied Meanings 
the analysis did not reveal any substantial differences between subscores in 
difficulty or any high correlations of subscores with predicted effectiveness. 
In the audlo-PONS, the analysis revealed only that the four Items in which the 
two answer options were similar in both ways (posltlvlty of feelings expressed 
and status of the speaker) were harder, in general, than the other items. In 
the vlsual-PONS, the body Items were somewhat more difficult than the face 
Items, and the Items In which the two answer option,? were similar in status of 
the speaker were slightly harder than those In which the answer options differed 
In this respect. None of the subscores on either PONS test correlated sub- 
stantially with predicted effectiveness. However, at School 2 the face Items 
subscore on the visual PONS correlated .34 with ratings of academic work; the 
correlation for the body items subscore was .17. The analysis of subscores on 

Photo Sequence Comprehension Test also revealed some differences in the 
difficulty of the different item sets, but no subtest correlated with predicted 
effectiveness at both schools. 

The analysis of subscores on the Photo Classification Teat yielded some 
interesting results. The second and third item sets were much easier than 
the others and, unlike the others, did not correlate well with predicted 



T«bl« 5. SubteMNi Mean percent corract and 

correlation with predicted ef fectlvoneee. 



l^teiC'^nd subtest 

PnC**^erson Perception Test 
^*j"*^clty of person pictured} 

IHock ■ 

^'^P^neee 
; Sex Of psraon pictured 

' '•■Mile 

Test Znplled Meanings 

; lew i^io 

ie«n« 11-20 

V It«w« 21-30 

^^OHS'«udio 

? }t0tho^ of construction: 

y l^^ndon-spllced 

y: Cont«nt-£lltered 

^; Si^^^trlty of options: 

^ Posit ivity end dominance 

V Posltivity only 

V Dominance only 
^ neither 



PONS^^aual 
Picture content 
pace 
Body 

Slffllsr^ty of opt lone s 
P08lf;^v£^y gnd donlnence 

posit; i^vlty only 
t'^'^^'.'iaiice only 
ftel^:her 



iPhoto Sequence Comprehension Test 



ie« eet 1 
Item set 2 
Item set 3 
Item set 4 
ie«» aet 5 



fPhotb Claaalflcatlon Test* 
Item aet 1 
Item Bet 2 
leew.aet 3. 
Item aet 4 
■ Item aet 5 



No. of Items 



10 
10 
10 
10 



20 
20 



10 
10 
10 



20 
20 

4 
10 
13 
13 



20 
20 

10 
10 
6 
14 



4 

4 
4 
4 
4 



Mftnn nercent correR t 
School 1 School 2 



Correletlon with predicted effectiveness 
School 1 School 2 



59 
58 
58 
64 

• 62 
.63 



64 
69 
63 



65 
66 

46 
66 
66 
66 



78 
70 



72 
75 
73 
77 



71 
SI 
86 
69 
SO 



72 
90 
93 

68 
57 



60 
63 
56 
62 

.62 
.59 



61 
67 
61 



66 
64 

43 
64 
70 
68 



80 
67 

72 
75 
70 
76 



72 
49 
82 
62 
38 



58 
80 
83 
45 
46 



-.05 
.06 

-•11 
.14 

.03 
.01 



.14 
.07 
-.10 



.00 
.05 

.07 
.07 
.07 
.03 



-.15 
.03 

-.09 
-.18 
-.02 
.08 



-.03 
-.02 
-.04 
.09 
.23 



• 30 

• 05 
-12 

• 41 

• 20 



• 07 
-•22 
-•01 

.12 

• 06 
-•13 



• 10 

• 06 

• 26 



-•08 
-•08 

-• 20 
-•33 

• 13 

• 14 



• 18 

• 01 

• 12 

• 16 

• 07 
-•02 



-•27 
•06 

-•02 
•05 

-05 



• 44 
.15 
•09 

• 54 

• 17 



*rhe correlations Involving subtesta of thla test do not include students who left all Items In the »ubt 



est blanks 



30 
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ef f ectlveness* The first and fourth Item sets correlated strongly with predicted 
effectiveness at both schools. Because of the speededness of this test, the 
correlations Involving the fourth and fifth Item sets were based on a self- 
selected subsample of the students that excluded the slower workers; persons who 
left all Items In an Item set blank were excluded In computing the correlation* 
Therefore, the correlations reported In Table 5 for these Item sets may be 
somewhat smaller than the correlations that would have been observed for these 
Item sets If they had been placed earlier In the test« 

One question of considerable practical Importance concerns the relationship 
between the nonverbal communication tests and commonly used academic aptitude 
tests: To what extent do the tests Investigated In the present study provide 
Information that Is not provided by the more commonly used tests? Some of the 
students participating In the present study had taken the Graduate Record 
Examinations (GRE), the Miller Analogies Test , or both. At School 1, scores 
were available for 42 students on the GR£ verbal and quantitative tests, for 
48 students on the GRE advanced test In psychology, and for 15 students on the 
Miller Analogies Test* The correlations between these tests, the nonverbal 
communication tests » and the faculty ratings of "academic work" and "predicted 
effectiveness" are shown in Table 6« The correlations among the three GRE 
scores are .60 (verbal with quantitative)^ .52 (verbal with advanced psychology) » 
and .41 (quantitative t;lth advanced psychology). Because of the small number 
of students taking the Miller Analogies » the correlations Involving this test 

should be regarded only as suggestive* No correlations were computed for 
School 2» where only 8 students had taken the GRE and only 8 had taken the 
Miller Analogies. 
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Table 6. Correlations with commonly used academic 
predictors. (School 1 only) 



Tests; 

Inter-Person 
Perception Test 

Test of Implied 
Meanings 

PONS-audlo 

PONS-vlsual 

Photo Sequence 

Comprehension Test 

'Photo Classification 
Test 

Group Embedded 
Figures Test 



Graduate Record Examinations 

Advanced 

Quantitative 
(n-A2) 



Verbal 
(n-A2) 



Psychology 
(n-48) 



.19 

.37 

.08 
-.05 
.11 

.05 

.38 



.07 

.33 

.00 
-.22 
.03 

-.05 

.49 



.04 

.07 

.21 
-.15 
.18 

.06 

.16 



Miller Analogic 
(n-15) 



-.29 

.52 

-.22 
.35 
.11 

.03 

.27 



Faculty ratings: 
"Academic work" 

"Predicted 

effectiveness" 



.31 

.16 



.24 
.11 



.59 
.39 



.49 
.56 



25 



Ih« corwlatlons In Table 6 Indicate that in general, the nonverbal 
conmunlcation testa showed quite low correlations with the more conventional 
predictors. The Photo Classification Taat . which was the only effective 
predictor at School 1, correlated essentially zero with all three GRE scores 
(and with Miller Analogies also). The visual PONS actually correlated nega- 
tively with all three GRE scores. Incidentally, the GRE advanced psychology 
score appears to be an effective predictor of faculty ratings at School 1 
(and the Miller An alogies Test mav also be an effective predictor). The GRE 
verbal and quantitative scores do not appear to be nearly as effective* even 
though they correlate fairly well with the GRE advanced psychology score. 
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DISCUSSION 



The results of the present study are mostly in the form of correlations 
computed from samples ranging in sice from approximately 40 persons to approxi- 
mately 80 persons. With data of this type, we must beware of over-interpretatii 
i*e«, attempting to explain "results'* that would not generalise to future 
samples or to the population we are interested in* However, when the same two 
variables correlate approximately ,25 in two completely independent samples, we 
are probably Justified in concluding that the relationship between them is 
real* And if the two independent samples represent different populations of 
students, faculty raters, and institutional settings, we have some grounds for 
expecting the result to generalize to other, similar populations. 
^ Why, then, was the Photo Classification Test the only one of the nonverbal 
communication tests to correlate with faculty ratings at both schools? At 
least two important features distinguish this test from the other tests used in 
the study. First, the test-taker's task is considerably more complex. Instead 
of simply choosing one picture or descriptive phrase from two or four options 
presented, the test taker must look at six highly similar photos that have been 
classified into two groups and determine the basis for the classification . 
Only then can the test«-taker correctly classify the remaining three photos. 
Second, the photos in this test were "stage directed" so that features that 
were irrelevant to the classification were deliberately varied in ways that 
did not consistently distinguish between the two groups of photos. That is, 
features of the photos that had nothing to do with nonverbal communication or 
with the feelings role-played by the persons in the photos were varied, but not 
In ways that would permit the test-taker to distinguish the Group A photos from 
the Group B photos on that basis* 
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The amaysla of aubscores from the Photo Clasaif ication Test provides some 
•vldanca for tha Importance of these Irrelevant features In the photos. Two 
of the f^ve item-seta were especially efrt^ctlve as predictors of the faculty 
ratlngSi An Inspection of the test reveals that In both these Item sets (and 
not In the other three), one of the Irrelevant features that was deliberately 
varied <»ccuple8 a prominent and conspicuous place In the photos. This fact 
suggests that the effectiveness of the test may be In Identifying persons who 
can perceive and Interpret nonverbal messages In the presence of distracting 
stimuli* 

The Instructions used for the Photo Classification Test In this study did 
not Inform the students that the basis for the classification had anything to 
do with nonverbal communication or with feelings and emotions. The sample Item 
used to Introduce the classification task was chosen so as not to help the 
students discover this fact. The photos In the sample Item simply showed a man 
at work In an office. In the "Group A" pictures he was seated at the desk; In 
the^Group B" pictures he was standing behind the desk. The effectiveness of 
the test may depend on the students' not knowing In advance that the test 
Involves nonverbal communications. If so, the test would be best used for 
guidances rather than selection, so that test-takers would have no reason to 
attempt to prepare for the test. In either case (guidance or selection), the 
Information gained from this test would be an addition to the Information 
provided by commonly used academic aptitude tests; there Is practically no 
overlap • 

While the Photo Classification Test showed positive correlations with most 
of the faculty ratings at one or both of the two schools. It predicted some 
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of the ratings much better than others. It seems to have been most effective 
at predicting "academic work," "clarity," "initiative," "organisation," and 
"predicted effectiveness." It seems to have been least effective at predicting 
interpersonal relations with persons other than faculty, "tact," and "warmth." 
This pattern of results suggests that the test may function as an indicator of 
the extent to which the test-taker is inclined and able to view interpersonal 
situations analytically. (The test does not seem to be simply a test of 
general analytic ability; its correlations with the Group Embedded Figures Test 
and the Graduate Record Examinationa were quite low.) 

The lack of positive results for the other five nonverbal communication 
tests is surprising and disappointing. Four of those tests were selected because 
they appeared to the author to be especially promising; the fifth was the 
author's own creation. The lack of success for the random-spliced subtest of 
the audio-PONS is particularly surprising in light of the positive results 
reported by McClelland and Fiske (1974). The visual-PONS was correlated with 
several of the faculty ratings at School 2 but not at School 1; it may be worth 
investigating further. 

The pattern of correlations between the Group Embedded Figures TPBt and the 
faculty ratings at School 2 reflects the pattern one would expect on the basis 
of the theory of field-dependence (Witkin, et al .. 1977). High scorers on this 
test tended to be rated higher in academic work, "clarity," and "organization" 
than low scorers, but lower in interpersonal relations with peers, "sensitivity," 
and "warmth." Witkin, et^. (1977) stated that ". . . there has not been a 
real check on the expected relation between field dependence and better per- 
formance in educational domains where a social orientation is emphasized." The 
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29 



re«uXt« of thU study tuggMt that the nature of this ralationship will depend 
heavily on how 'l>etter performance" la defined. If it ia defined in terms of 
academic work, the field-independent students , who score high on Embedded 
F-lnures, are likely to do better than the field-dependent students. If it is 
defined in terms of such personal characteristics as "enthusiasm," "sensitivity, 
and "warmth," the field-dependent students are likely to do better^ 
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